1. Field of the Invention
The present invention relates generally to handheld computing devices and methods of displaying 3D content. More specifically, the invention relates to a handheld computing device and techniques for displaying 3D content and mixed reality content in a realistic, spatially correct manner.
2. Description of the Related Art
The amount of three-dimensional content available on the Internet and from other sources, such as video games, medical imaging, and movies, is rapidly increasing. Consumers are becoming more accustomed to hearing about “3D” in various contexts, such as movies, TV, games, and online virtual environments. However, mobile or handheld devices have so far not adapted to enabling users to view and interact with 3D content in a realistic way.
Current handheld devices have displays that are capable of rendering 3D content, however, they do not take into consideration two important factors. One factor is the device's location and orientation (relative to the physical environment around it). Another factor is the user's location and orientation (relative to the device). Because handheld devices currently do not consider these two factors, problems arise when displaying spatial 3D content, for example, a virtual house in a virtual environment or visual information overlay of an augmented reality system. In one scenario a user expects 3D content to be stationary on the display but instead it appears to be moving from the user's perspective. It should be stationary on the display relative to the physical environment when the user is stationary and is only moving the handheld device. In this case, the spatial content should not appear to be moving. In another scenario, the spatial content is stationary when the user expects it to move. This occurs when the handheld device is stationary and the user moves (e.g., shifts head left to right). In this case, the user expects the spatial content to move on the screen because the angle at which the user is viewing it is changing. However, with current handheld devices, the 3D content does not move. In both these instances, the rendering of the spatial content is not realistic or natural and there is a breakdown in the interaction.
It is desirable to make the visual experience of the 3D content on a 2D display more realistic and immersive. Currently, handheld devices may take into account view-dependent rendering (VDR) only, which assumes that the device is stationary and that only the user's head moves in reference to the display. Other handheld devices may take into account position-dependent rendering (PDR) only, which assumes that the user's head and device move synchronously (i.e., are “locked”) and also assumes that the device (or, specifically, the display) is on-axis (on-center) of the user's view. Neither of these features (PDR or VDR) alone provides for a realistic and spatially correct rendering of 3D content or of 3D content and physical image composites.